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(54) IMAGE ENCODER AND ITS METHOD 

(57) An image encoder and its processing method 
for compression-encoding a dynamic image at a varia- 
ble encoding rate in the case of an application required 
to perform real-time processing. A variable bit rate 
(VBR) control section (22) determines a VBR control 
quantizing scale VBRq according to a VBR buffer value 
held in a VBR buffer recording section (21). A constant 
bit rate (CBR) control section (23) determines a CBR 
control quantizing scale CBRq according to an inputted 
maximum bit rate BRmax. A quantizing scale compar- 
ing section (25) outputs the CBRq during constant rate 
encoding and outputs the larger one of the VBRq and 
CBRq during variable rate encoding. An image com- 
pression-encoding section (24) encodes the inputted 
image signal at a quantizing scale outputted from the 
quantizing scale comparing section (25) and outputs a 
bit stream(S20). 
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Description 

TECHNICAL FIELD 

[0001] The present invention relates to an appara- 5 
tus for encoding an image, more particularly relates to 
an image encoder capable of encoding a moving picture 
at a variable bit rate and a method of the same. 

BACKGROUND ART 

[0002] In a general image encoding apparatus, the 
digitized image signal is encoded at a constant bit rate 
or at a variable bit rate. A bit stream encoded at a con- 
stant bit rate has a constant bit rate and does not depart 
much from an object bit rate. As opposed to this, a bit 
stream encoded at a variable bit rate does not have a 
constant bit rate and varies in accordance with the char- 
acteristics of the image signal to be encoded. Note that 
in this case, compression and encoding are performed 
on an image signal within a range not exceeding a max- 
imum bit rate subscribed by standards. 
[0003] When encoding a digitized moving picture 
signal, encoding bit rate control, that is, control so as to 
give a desired bit rate when viewing a certain time span 
of a coding series generated by encoding, is performed. 
A plurality of standards for compressing and encoding 
an image signal changing over time, a so-called motion 
picture, at a high efficiency have been proposed hith- 
erto. Among them, the MPEG2 standard has currently 
gathered attention and come into practical use in a vari- 
ety of fields of application such as communication, 
broadcasting, and media storage. Digital versatile disks 
(DVD), one of the applications, is expected to be 
increasingly widely used in the future as recording 
media for compressing, encoding, and storing motion 
pictures based on the MPEG2 standard due to its large 
capacity, low cost, and ease in handling. 
[0004] The MPEG2 standard does not define the 
above encoding control for control of the bit rate. There- 
fore, it is sufficient that the amount of encoded informa- 
tion be within a certain range subscribed by standards. 
Normally, a variable bit rate is used in authoring DVDs 
(editing work for preparing a DVD title). The encoding 
method at this time investigates the encoding difficulty 
of the source as a whole in a first encoding and per- 
forms encoding assigning an amount of code corre- 
sponding to the encoding difficulty in the second 
encoding to realize a variable bit rate. As a result, a 
larger code amount is assigned to an image signal hav- 
ing a high difficulty, while a code amount assigned to an 
image signal having a low difficulty is suppressed and 
therefore uniformity of the image quality can be 
attained. 

[0005] In an application recording a motion picture 
signal obtained by an image pickup like a video camera 
on a recording medium in real time, the successively 
input motion picture signal has to be compressed, 



encoded, and recorded sequentially. In such a case, 
since the entire image to be encoded cannot be investi- 
gated in advance, the image is normally compressed 
and encoded at a constant bit rate using a fixed code 
amount for every GOP (group of pictures) in units of, for 
example, about 15 frames. 

[0006] Figure 7 is a view of a view of an encoding 
rate when encoding an image signal at a constant bit 
rate. As shown in Fig. 7, when performing compression 
encoding on an image signal at a constant bit rate, a bit 
rate of an obtained coding series is made constant to be 
almost an object bit rate regardless of the characteris- 
tics of the image. To realize the constant bit rate, a bit 
rate control portion of the image encoder rapidly 
changes a quantizer scale in units of macroblocks com- 
prised of a several blocks for the object bit rate given in 
advance. 

[0007] The compression and encoding by a con- 
stant bit rate make the bit rate constant regardless of 
the characteristic of the image signal, therefore the pic- 
ture quality is good in a scene wherein the encoding dif- 
ficulty is low, but there is a disadvantage that the picture 
quality deteriorates in a scene wherein the encoding dif- 
ficulty is high. 

[0008] For this reason, it is desired that a bit rate is 
set variable so that the code amount obtained in com- 
pression and encoding falls in a range of a prescribed 
value while maintaining the picture quality to be not less 
than a fixed level. Figure 8 is an example of a bit rate of 
a coding series obtained by variable bit rate control. In 
the variable bit rate control, the quantizer scale is con- 
trolled for example in units of GOPs fro an object bit 
rate. Namely, a response speed of the quantizer scale is 
made slow. As a result, as shown in Fig. 8, the bit rate 
becomes low in portions where encoding difficulty is 
low, while the bit rate becomes high in portions where 
encoding difficulty is high. By changing the bit rate over 
time in accordance with the encoding difficulty, picture 
quality of a reproduced image can be maintained to be 
almost uniform. 

[0009] In an application like a video camera 
wherein an image processing is required in real time, 
however, the characteristics of the entire image signal 
cannot be investigated in advance. Therefore, the bit 
rate is set based on experience. Since images obtained 
from an actual image pickup device changes from 
moment to moment and the encoding difficulty also 
changes over time, the encoding bit rate exceeds a 
maximum bit rate designated at a sequence header at 
the top of a bit stream in the portions where encoding 
difficulty is high in some cases. Therefore, it is impossi- 
ble to generate a bit stream based on standards in all 
cases merely by making the response speed of the 
quantizer scale in accordance with changes of encoding 
difficulty. 

[0010] Furthermore, when considering an applica- 
tion to consumer goods, a simple algorithm is desired 
by which a circuit scale of a hardware and processing 
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amount of software are not increased much. 
DISCLOSURE OF INVENTION 

[0011] The present invention was made in consider- 
ation with such a circumstance and has as an object 
thereof to provide an image encoder and processing 
method capable of realizing compression and encoding 
of a motion picture at a variable bit rate in an application 
where real time processing is required. 
[0012] To attain the above object, an image encoder 
of the present invention claims*. 

BRIEF DESCRIPTION OF DRAWINGS 

[0013] 

Figure 1 is a circuit diagram of an image encoder 
according to an embodiment of the present inven- 
tion. 

Figure 2 is a block diagram of concepts of an image 

encoder of the present invention. 

Figure 3 is a block diagram of the configuration of 

an encoder in an image encoder of the present 

invention. 

Figure 4 is a block diagram of the configuration of 
an image compression encoder of the encoder 
shown in Fig. 2. 

Figure 5 is a flow chart of an operation of an image 
encoder of the present embodiment. 
Figure 6 is a view of an example of an image encod- 
ing bit rate in an image encoder of the present 
embodiment. 

Figure 7 is an example of an image encoding bit 
rate by constant bit rate control of the related art. 
Figure 8 is a view of a case of exceeding a maxi- 
mum bit rate subscribed by standards due to char- 
acteristics of an image in variable bit rate control of 
the related art. 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0014] Figure 1 is a view of the configuration of an 
embodiment of an image encoder according to the 
present invention. 

[0015] As shown in the figure, an image encoder of 
the present invention comprises a control circuit 10, an 
encoder 20, and an image input device 30. 
[0016] The control circuit 10 respectively outputs to 
the image input circuit 30 and the encoder 20 control 
signals S12 and S13 in accordance with a control signal 
S1 input from the outside and controls their operations. 
[0017] The image input device 30 outputs image 
data S30 obtained by digitizing the image signal to the 
encoder 20. The image input device is any pickup such 
as a video camera or image reproducing apparatus for 
reproducing an image signal from another recording 
medium and need only have the function of generating 
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an image signal by some means and converting an 
image signal in an analog format to a digital signal. 
[0018] The image input device 30 generates image 
information data S32 in accordance with a type, charac- 

5 teristics, etc. of the input image signal and supplies the 
same to the control circuit 10. The control circuit 10 
receives the image information data S32 and outputs a 
compression and encoding parameter S12 to the 
encoder 20 in accordance with the characteristics etc. 

10 of the input image signal. Note that the compression 
and encoding parameter S12 is comprised of a plurality 
of parameters or signals such as an object bit rate, max- 
imum bit rate, and variable bit rate instruction signal 
which will be explained later on. 

15 [0019] The encoder 20 performs, in accordance 
with the compression and encoding parameter S12 
from the control circuit 10, compression and encoding 
on the digital image data S30 sent from the image input 
device 30 based on a predetermined algorithm and out- 

20 puts a bit stream S20. Note that as a result of compres- 
sion and encoding of the image data by the encoder 20, 
the amount of data of the bit stream S20 is greatly 
reduced compared with the image data S30 before 
being subjected to the compression and encoding. As a 

25 result, the transfer capacity of a channel or a storage 
capacity of an image recording medium can be greatly 
reduced by transferring or storing of information of the 
bit stream S20 compressed and encoded by the 
encoder 20. 

30 [0020] In the encoder 20 of the present embodi- 
ment, by using a so-called variable bit rate for control- 
ling a bit rate at the time of compressing and encoding 
an image signal in accordance with the input image 
data, an appropriate bit rate can be set in accordance 

35 with characteristics of the image data and a uniform pic- 
ture quality of the entire image and the compression 
efficiency can be improved. 

[0021] Figure 2 is a block diagram of the configura- 
tion of the encoder 20 in the present embodiment. As 

40 shown in Fig. 2, the encoder 20 comprises a variable bit 
rate control circuit 22a, a constant bit rate control circuit 
23a and a quantizer scale select circuit 25a. 
[0022] The variable bit rate control circuit 22a is a 
first quantizer scale setting means for setting a first 

45 quantizer scale for performing image encoding in a first 
image processing unit and setting a gently changing 
first quantizer scale QS1 in accordance with encoding 
difficulty of an input image signal in units of a first image 
processing unit, for example, in units of GOPS com- 

50 prised of a plurality of pictures with respect to an object 
bit rate. 

[0023] The constant bit rate control circuit 23a is a 
second quantizer scale setting means for setting a rap- 
idly changing second quantizer scale QS2 in order not 
55 to exceed a maximum bit stream with respect to the 
maximum bit stream designated by a sequence header 
at the top of the bit stream in units of a second image 
processing unit, for example, in units of pictures or mac- 
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roblocks, which is shorter than said-first image process- 
ing unit (GOP) 

the quantizer scale select circuit 25a compares the 
first quantizer scale QS1 set by the variable bit rate 
control circuit 22a with the second quantizer scale 
QS2 set by the constant bit rate control circuit 23a 
and outputs the larger quantizer scale as a quan- 
tizer scale QSC tor performing encoding processing 
on the input image signal. 

[0024] As explained above, the image encoder of 
the present embodiment obtains a gently changing first 
quantizer scale QS1 in units of long image processing 
unit, for example, in units of GOPs with respect to an 
object encoding bit rate in accordance with encoding 
difficulty of an input image, obtains a quickly changing 
second quantizer scale QS2 in units of short image 
processing units, for example, in units of macroblocks 
with respect to a maximum bit rate prescribed by stand- 
ards, furthermore, selects the larger of the two as a 
quantizer scale QSC for performing encoding on the 
input image. 

[0025] By setting the quantizer scale QSC as 
above, an input image signal can be encoded at an 
encoding bit rate close to the object bit rate in average 
corresponding to the encoding difficulty of the input 
image signal, and it becomes possible to guarantee not 
to exceed the maximum bit rate prescribed by the stand- 
ards. 

[0026] Figure 3 is a view of a specific example for 
realizing the image encoding processing of the present 
invention and is a block diagram of the configuration of 
an encoder 20. As shown in Fig. 3, the encoder 20 com- 
prises a VBR buffer memory 21, VBR controller 22, 
CBR controller 23, image compression encoder 24, and 
quantizer scale comparator 25. Below, the components 
of the encoder 20 and operations of the encoder 20 as 
a whole will be explained. 

VBR Buffer Memory 21 

[0027] The VBR buffer memory 21 holds a parame- 
ter VBRb called a VBR buffer value (hereinafter referred 
to as a VBR buffer value) and is provided for making the 
amount of deviation from an object bit rate in a series of 
encoding processing small in total value. Before starting 
the processing, the VBR buffer memory 21 stores a 
VBR buffer initial value INI input from the control circuit 
1 0 shown in Fig. 1 . The simplest method of initializing is, 
before starting the processing, to initialize in accord- 
ance with an initialize control signal from the control cir- 
cuit 10 and to set the initializing value to be "O" or a 
value given in advance. 

[0028] After starting compression and encoding, 
the initial value INI stored in the VBR controller 20 is 
provided, the VBR buffer value VBRb is input from the 
VBR controller 22 at the time of completing the encod- 
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ing, which is stored, and the stored VER buffer value 
VBRb is output to the VBR controller 22 in the next 
encoding. 

5 VBR Controller 22 

[0029] The VBR controller 22 sets a VBR quantizer 
scale VBRq in accordance with a VBR response value 
VRT, object bit rate BR ob j, and maximum bit rate BR max 

w input from the control circuit 1 0 and outputs the same to 
the quantizer scale comparator 25. 
[0030] The VBR controller 22 has as internal 
parameters the VBR buffer value VBRb for obtaining an 
output VBR quantizer scale, a remaining bit quantity 

/5 RBG in a GOP at the time of encoding at the object bit 
rate, and a response speed VBRr indicating a response 
speed of the VBR quantizer scale. 
[0031] In the VBR controller 22, the VBR quantizer 
scale VBRq can be obtained from the equation below. 

20 

VBRq=VBRb/(2 n • VBRr) (1) 

[0032] In equation (1), n is a calculation parameter 
set in accordance with an application etc. and is 

25 assumed for example to be n=9. 

[0033] When the VBR quantizer scale VBRq 
becomes "1", the generated bit quantity abruptly 
increases and VBR rate control becomes unstable, thus 
countermeasures such as forcibly changing the VBRq 

30 to be "2° are taken when the VBRq calculated from 
equation (1) is not more than "1 ". 
[0034] Furthermore, by taking note of the difference 
of the l/P/B pictures, a value obtained by multiplying the 
value obtained from equation (1) with 1.4 is made the 

35 quantizer scale VBRq when the coding type is a B-pic- 
ture. Namely, when in the case of a B-picture, the quan- 
tizer scale is always set to be quantizer scales of I- and 
P-pictures multiplied with 1.4. This is on an assumption 
that when assigning an encoding amount saved in a B- 

40 picture by encoding the B-picture a little larger compar- 
ing with the I- and P-pictures to encoding amounts of I- 
and P-pictures, the S/N of the I- and P-pictures is 
improved and the S/N of the B-picture generated by 
referring thereto is also improved, so the S/N is totally 

45 improved. 

[0035] In the VBR controller 22, the quantizer 
scales of the l/P/B-pictures are determined in a simple 
method as explained above, but by inspecting the 
encoding difficulties of the respective pictures, the 

so quantizer scales can be made different in accordance 
therewith. 

[0036] An object bit quantity for every GOP is set in 
the remaining bit quantity RBG in the GOP. Then, for 
example when a generated bit quantity for every MB 
55 (macro block) is received, the amount is subtracted from 
the object bit quantity. The amount of deviation from the 
object bit quantity remains in the RBG at the end of 
each GOP, thus, which is subtracted from the VBR 
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buffer value VBRb. At this time, when the VBR buffer 
value VBRb becomes negative, it is set to be "0". Also, 
when the VBR buffer value VBRb comes close to over- 
flowing, it is fixed to be a maximum value. 
[0037] As explained above, the initial value of the 
VBR buffer value VBRb at the time of starting encoding 
is input from the VBR buffer memory 21. Here, for 
example, when the VBR buffer value input from the VBR 
buffer memory 21 is not "0", the value is used, while 
when it is "0", a value obtained from the equation below 
is used. 

VBRb=m-VBRr/(BR obj -2 k ) (2) 

[0038] In equation (2), m and k are calculation 
parameters. Here, for example m=2.5x10 7 and k=10. 
[0039] As the response speed VBRr in equation (2), 
a value supplied from the control circuit 1 0 shown in Fig. 
1 is used. For example, when the response speed VBRr 
is set at about 200, a response speed of tens of sec- 
onds is obtained, while when set at about 4000, a 
response speed of tens of minutes is obtained. When 
completing the encoding, the VBR buffer value VBRb 
calculated from equation (2) is transferred to the VBR 
buffer memory 21 and stored therein. 
[0040] A value obtained in the equation below is set 
to the remaining bit quantity RBG in the GOP at first. 

RBGsBR^j • N/PCR (3) 

[0041] In equation (3), N is the interval of l-pictures 
and PCR is a picture rate. 

CBR Controller 23 

[0042] The CBR controller 23 sets a CBR quantizer 
scale CBRq in response to the object bit rate BR obj , 
maximum bit rate BR max , and VBR instruction signal 
VBD input from the control circuit 10 and outputs the 
same to the quantizer scale comparator 25. 
[0043] In the present embodiment, the CBR control- 
ler 23 performs, for example, rate control of the CBR 
based on an algorithm of a test mode 5 (TM5) pre- 
scribed by the MPEG2 standards. 
[0044] In TM5, image encoding processing is per- 
formed in units of GOPs. A bit amount R to be assigned 
to a GOP is determined from the number of pictures and 
bit rates included in every GOP. A remaining amount AR 
(positive or negative) of the bit amount in the previous 
GOP of the GOP being processed is added to the 
amount of on-going processing. Also, an object bit 
amount for a picture to be encoded from now is deter- 
mined from the bit amount R for every picture and 
encoding difficulty of each of the picture types (l/B/P). 
Then, a bit amount actually generated after completing 
the encoding is subtracted from the object bit amount 
and a remaining amount AR of the bit amount is calcu- 
lated. 



[0045] In processing in units of macroblocks (MB), 
pictures are encoded in units of MBs, and when the 
generated bit amount comes close to exceed the object 
bit amount in every MB, the quantizer scale is set to be 

5 large, while when the generated bit amount comes 
close to be under the object bit amount, the quantizer 
scale is set to be small. Furthermore, considering the 
fact that human visual characteristics is sensitive to 
deterioration of picture quality on a flat portion in a 

ro screen, the quantizer scale is set to be large in a portion 
where an image is relatively flat, while the quantizer 
scale lis set to be small in a fine image portion. 
[0046] Note that the image encoder of the present 
invention is not limited to performing rate control in CBR 

15 based on an algorithm of TM5, and may perform the 
rate controlling based on other control format. 
[0047] The CBR controller 23 in the image encoder 
of the present embodiment is a CBR controller installed 
in an MPEG encoder of the related art being improved. 

20 The CBR controller 23 of the present embodiment is 
additionally given the functions below. 
[0048] At first, the CBR controller 23 controls the 
CBR quantizer scale 1 in accordance with the VBR 
instruction signal VBD input from the control circuit 1 0 

25 shown in Fig. 1 . Specifically, when the VBR instruction 
signal is "0", the rate is controlled based on the object 
bit rate BR ob j, while when the VBR instruction signal 
VBD is "1 ", the rate is controlled based on the maximum 
bit rate BR max . Namely, when the VBR instruction signal 

30 VBD is "0°, normal CBR rate control is performed and 
the quantizer scale is controlled so that the bit rate of a 
bit stream obtained by the compression and encoding 
does not deviate with respect to the object bit rate 
BR ob j. On the other hand, when the VBR instruction sig- 

35 nal VBD is "1 ", the VBR rate is controlled and the quan- 
tizer scale is controlled so that the bit stream obtained in 
the compression and encoding does not exceed the 
maximum bit rate BR max . 

[0049] Next, the next GOP's object bit rate R is 

40 determined in the TM5 of the MPEG2 for every GOP. At 
the end of the GOP, the R is subtracted from a bit quan- 
tity generated in the GOP. After that, this is deemed the 
remaining bit quantity of the GOP and indicated by AR. 
Then, the remaining bit quantity AR is added to the 

45 object bit quantity R of the next GOP, but when the VBR 
instruction signal VBD is "1 ", the processing is not per- 
formed. This is because when the VBR instruction sig- 
nal VBD is "1°, the quantizer scale is controlled by the 
VBR rate control and the object bit quantity R is left at 

so the end of the next GOP in the most cases (namely, 
AR>0). When adding this to the next GOP's object bit 
quantity R, it could end up in overflow and when an 
image exceeding the maximum bit rate is input, the 
quantizer scale is not raised, and a VBV (video buffering 

55 verifier) ends up in underflow. 

[0050] The next GOP's object bit rate is determined 
for every GOP in the TM5 of the MPEG2. Due to the 
above additional functions, the CBR controller 23 of the 
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present embodiment performs three different kinds of 
processing as described below in accordance with the 
input VBR instruction signal VBD, and the next GOP's 
object bit quantity R is determined. 

1. When the VBR instruction signal VBD=0, the 
next GOP's object bit quantity R i+1 is calculated 
from the next equation in accordance with the 
present GOP's remaining bit quantity ARj. Here, i 
indicates the number of the GOP. 

R^BR^'N/PCR+ARj (4) 



Note that in equation (4), N is the interval of I- 
pictures and PCR is a picture rate the same as in 
the above equation (3). 

2. When the VBR instruction signal VBD=1 and the 
remaining bit quantity ARj of a GOP immediately 
before the GOP is negative (ARj<0), the next GOP's 
object bit quantity R i+1 is calculated from the next 
equation. 

R t+1 max * N/PCR+AR | (5) 

3. When the VBR instruction signal VBD=1 and the 
remaining bit quantity ARj of a GOP immediately 
before the GOP is positive or "0" (ARj>0), the next 
GOP's object bit quantity R )+1 is calculated from the 
next equation. 

Rk1=BR m ax'N/PCR (6) 

[0051 ] The above case 1 is the same processing as 
the normal processing of the TM5 of the MPEG2. The 
CBR is controlled. The remaining bit quantity AR for 
every GOP is always fed back to the next GOP's object 
bit quantity R i+1 , which is corrected in accordance with 
the remaining bit quantity AR ( of the immediately before 
GOP. 

[0052] In the cases 2 and 3, the VBR is controlled 
and the next GOP's object bit quantity is calculated in 
accordance with the remaining bit quantity AR of every 
GOP so as not to exceed the maximum bit rate BR max . 
When the remaining bit quantity ARj is negative in the 
GOP immediately before, as shown in equation (5), the 
remaining bit quantity AR| is fed back to the next GOP's 
object bit quantity in the GOP immediately before and 
the next GOP's object bit quantity R M is corrected to be 
less. On the other hand, when the remaining bit quantity 
ARj of the GOP immediately before is positive or "0", the 
remaining bit quantity ARj of the GOP immediately 
before is not fed back for the above reason, and the next 
GOP's object bit quantity R i+ i is determined in accord- 
ance with the maximum bit BR max etc. as shown in 
equation (6). 

[0053] The CBR controller 23 obtains the CBR 
quantizer scale CBRq based on the GOP's object bit 



quantity calculated as above and supplies the same and 
the VBR quantizer scale VBRq obtained in the VBR 
controller 22 to the quantizer scale comparator 25. 

5 Quantizer Scale Comparator 25 

[0054] The quantizer scale comparator 25 calcu- 
lates a base quantizer scale QSC in accordance with 
the VBR quantizer scale VBRq, CBR quantizer scale 
10 CBRq, and VBR instruction signal VBD input from the 
CBR controller 23 and supplies the same to the image 
compression encoder 24. 

[0055] For example, when the VBR instruction sig- 
nal VBD is "0", the quantizer scale comparator 25 out- 

15 puts the CBR quantizer scale CBRq to the image 
compression encoder 24. As a result, since the bit rate 
is controlled based on the normal CBR rate control, the 
quantizer scale is controlled so that the bit rate does not 
always largely depart from the object bit rate BR ob j. 

20 [0056] On the other hand, when the VBR instruction 
signal VBD is "1", the quantizer scale comparator 25 
compares the VBR quantizer scale VBRq with the CBR 
quantizer scale CBRq and outputs the larger to the 
image compression encoder 24. As a result, the bit rate 

25 is controlled based on the VBR rate control, and the 
quantizer scale is controlled so that the bit rate does not 
exceed the maximum bit rate BR max . 

Image Compression Encoder 24 

30 

[0057] The image compression encoder 24 com- 
presses and encodes the image data S30 input from the 
image input device 30 shown in Fig. 1 in accordance 
with the object bit rate BRobj, maximum bit rate BR max , 

35 VBR instruction signal VBD, and quantizer scale QSC 
supplied from the quantizer scale comparator 25 and 
outputs a bit stream BSM. Here, the bit stream BSM is 
identical with an output signal S20 from the encoder 20 
in Fig. 1. Furthermore, the image compression encoder 

40 24 generates a bit quantity BQT indicating a bit quantity 
generated by the compression and encoding and a PCT 
signal indicating a picture coding type and outputs them 
to the VBR controller 22 and the CBR controller 23, 
respectively. 

45 [0058] Figure 4 is a block diagram of an example of 
the configuration of the image compression encoder 24. 
Below, the configuration and operations of the image 
compression encoder 24 will be explained with refer- 
ence to Fig. 4. 

so [0059] As shown in the figure, the image compres- 
sion encoder 24 comprises an adder 201 , a DCT por- 
tion 202, a quantizer 203, a variable length coder (VLC) 
204, an inverse quantizer 205, an inverse DCT portion 
206, an adder 207, a frame memory 208, a motion 

55 detector 209, a motion compensator 21 0, a switch cir- 
cuit 21 1 , and encoding controller 212. 
[0060] The difference of the image signal S30 input 
to the image compression encoder 24 and a reproduc- 
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tion image signal input from the switch circuit 211 is 
obtained in the adder 201 and output to the DCT portion 
202. During an intra-picture (l-picture) mode, a mean- 
ingful image is not input from the switch circuit 21 1 (the. 
output from the switch circuit during this mode is "0" in 5 
Fig. 4), so the image data input to the adder 201 is out- 
put as it is to the DCT portion 202. During a motion com- 
pensation prediction mode, image data based on an 
image reproduced before that is input from the switch 
circuit 211, so the difference is obtained in the adder w 
201 and the error is input to the DCT portion 202. 
[0061] The DCT portion 202 performs discrete 
cosine transform (DCT) on the image signal input from 
the adder 201 to reduce spatial redundancy. A trans- 
form coefficient obtained by the DCT is input to the 75 
quantizer 203. 

[0062] The quantizer 203 quantizes the transform 
coefficient obtained from the DCT portion 202 in 
accordance with the quantization rate input from the 
controller 21 3 and outputs it to the variable length coder 20 
204 and inverse quantizer 205. Note that the quantizer 
scale of the quantizer 203 is controlled by the encoding 
controller 212. 

[0063] The variable length coder 204 encodes the 
quantized transform coefficient to a variable length to 25 
generate an encoded image bit stream S20 and succes- 
sively outputs the same. 

[0064] The encoding controller 212 controls the 
quantization rate of the quantizer 203, recording of 
image data to a frame memory 208, switching of the 30 
switch circuit, etc. As shown in the figure, the encoding 
controller 212 sets a predetermined quantizer scale to 
the quantizer 203 in accordance with the object bit rate 
BR ob j input from the control circuit shown in Fig. 1 , max- 
imum bit rate BR max , VBR instruction signal VBD, and 35 
quantizer scale QSC from the quantizer scale compara- 
tor 25 shown in Fig. 3 and controls the quantization rate. 
[0065] The transform coefficient quantized by the 
quantizer 203 is input to the inverse quantizer 205. The 
transform coefficient is inversely quantized in the 40 
inverse quantizer 205 and further subjected to an 
inverse discrete cosine transform (inverse DCT) in the 
inverse DCT portion 206. When in a mode of motion 
compensation prediction, the inversely DCT'ed image 
signal and an image signal obtained by the above 45 
motion compensation prediction are added in the adder 

207 to be returned to the original image signal and 
stored in the frame memory 208. When in an intra-pic- 
ture mode, the image signal inversely DCTed in the 
inverse DCT portion 206 is stored as it is in the frame so 
memory 208. 

[0066] When performing the motion compensation 
prediction, the image signal stored in a frame memory 

208 is used. Namely, motion is detected in the motion 
detector 209 from the next image source signal to 55 
detect the motion vector based on the image signal 
stored in the frame memory 208. Furthermore, based 

on the motion vector, motion compensation prediction is 



performed in the motion compensator 210. Then, the 
image signal obtained by the motion compensation pre- 
diction is output to the adder 201 via the switch circuit 
21 1 and the difference of the input image source signal 
is obtained. 

[0067] The image compression encoder 24 having 
the above configuration encodes the input image signal 
S30 to generate a bit stream. When the VBR instruction 
signal VBD is "0", an object bit rate BRobj is output to 
the bit rate of the sequence header, while when the VBR 
instruction signal VBD is V, the maximum bit rate 
BRmax is output to the bit rate. When a quantizer scale 
is set for each of the macro blocks (MB), based thereon, 
the quantizer scale is changed aiming at the visual 
effect based on step 3 of bit rate control in the TM5 of 
the MPEG2 and the changed quantizer scale is used for 
quantization. Then, a bit stream generated for every MB 
is output. 

[0068] Also, the image compression encoder 24 
has the general function of an image encoding device in 
the MPEG2 of preventing a VBV from underflowing by 
forcibly cutting a DCT coefficient when the bit rate con- 
trol by using the quantizer scale is not performed well. 
[0069] Due to the above image compression 
encoder 24, it is possible to realize encoding of an 
image signal at a variable bit rate by one encoding oper- 
ation, that is, by one path processing. Therefore, for 
example in an image processing apparatus picking up 
and recording an image in real time such as a video 
camera, image encoding at a variable bit rate can be 
realized by one encoding operation, and uniformity of 
picture quality can be realized in an image signal having 
different characteristics, and furthermore the encoding 
efficiency can be improved by variable bit rate control. 
[0070] Figure 5 is a flow chart of the operation of an 
image encoder of the present embodiment. Below, the 
operation of the image encoder of the present embodi- 
ment will be explained by using specific examples of 
parameter values with reference to Fig. 5. 
[0071 ] Figure 5 is a flow chart of the operation when 
encoding an image at a variable bit rate, namely, a flow 
chart of image encoding in the case where the VBR 
instruction signal VBD in Fig. 3 is "1°. When the VBR 
instruction signal VBD is "0°, the image is encoded at a 
normal constant bit rate, the next GOP's object bit quan- 
tity is calculated only in the CBR controller 23 from 
equation (4) in accordance with the object bit rate BR ob j 
and the remaining bit quantity ARj in the GOP immedi- 
ately before, and in accordance therewith, an encoding 
scale in the next GOP is controlled. The image is gener- 
ally encoded based on the constant bit rate in an image 
encoder to which the normal MPEG2 standard is 
applied, so a detailed explanation of the operation will 
be omitted here. 

[0072] As shown in Fig. 5, after starting the encod- 
ing, first, the initial value of the VBR buffer value VBRb 
is obtained from equation (2) in accordance with given 
parameters (step S1). Here, for example, when 
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BR ob j=4Mbps, maximum bit rate BR max =8Mbps, VBR 
response speed VBRr=300, further, m=2.5x10 7 and 
k=10 in equation (2), the VBR buffer value VBRb 
becomes (2.5x1 0 7 x300/(4x10 6 x2 10 ))=1 92000 0. 
[0073] Next, as shown in step 2, the object bit quan- 
tity of the GOP is obtained in the VBR controller 22 
based on equation (3). Note that in equation (3), when 
assuming that GOPs are units of 15 picture, N=15 
stands. Furthermore, for example when encoding a tel- 
evision image signal in an NTSC format, the respective 
fields are updated at 29.97 Hz and the picture rate PCR 
at this time becomes 29.97 in this case. Therefore, the 
GOP's object bit rate becomes 
(4x1 0 6 x1 5/29.97)=2002002. 

[0074] Then, as shown in step S3, the GOP object 
bit rate is obtained in the CBR controller 23 in accord- 
ance with the maximum bit rate BR max . Here, the object 
bit quantity of the CBR becomes 
(8x10 6 x15/29.97)=4004004 in accordance with the 
maximum bit rate BR max and number of pictures N and 
picture rate PCR of the GOP. 
[0075] Next, the processing shown in step S4 is 
performed. Namely, the VBR quantizer scale VBRq is 
obtained from equation (1) based on the VBR buffer 
value VBRb obtained above. In equation (1), when 
assuming n=9, the VBR quantizer scale VBRq becomes 
(0920000/(2 9 x300))=12.5. Note that since an integer is 
required for the quantizer scale, the variable quantizer 
scale VBRq=13 is set here. 

[0076] When the picture coding type is 1 or 2, that 
is, in a case of an I- or P-picture, the variable quantizer 
scale VBRq=13 is set, while when the picture coding 
type is 3, that is, in a case of a B-picture, the variable 
quantizer scale is set at a value of a quantizer scale cal- 
culated from equation (1) multiplied with 1.4. Namely, in 
the case of a B-picture, VBRq=18 is set. 
[0077] Next, as shown in step S5, the CBR quan- 
tizer scale CBRq is obtained in the CBR controller 23 in 
accordance with the maximum bit rate BR max . 
[0078] The processing shown in step S6 is per- 
formed in the quantizer scale comparator 25. the VBR 
quantizer scale VBRq and the CBR quantizer scale 
CBRq are compared and the larger, as a. result of the 
comparison, is output to the image compression 
encoder 24 (steps S7 and S8). 
[0079] The image compression encoder 24 
encodes the input image signal in units of 1 MB by using 
the quantizer scale provided from the quantizer scale 
comparator 25 (step S9). Then, as shown in step S10, a 
bit quantity generated from the encoding is subtracted 
from the set GOP object bit quantity. The processing 
from steps S4 to S10 are repeated until encoding all of 
the image signal of 1 GOP. After completing the encod- 
ing on 1 GOP, a remaining bit quantity RBG of the 1 
GOP is taken out in step S12. The VBR buffer value is 
updated in the VBR controller 22 in accordance with the 
remaining bit quantity RBG (step S13). 
[0080] A remaining bit quantity AR of the GOP is 



calculated in accordance with the generated bit quantity 
in the CBR controller 23, and in accordance therewith, 
an object bit quantity of the GOP in the CBR controller 
23 is updated based on equations (5) and (6) (step 

5 S14). As explained above, when the remaining bit quan- 
tity AR in the GOP immediately before is negative, the 
next GOP's object bit quantity is calculated based on 
equation (5), while when the remaining bit quantity AR 
in the GOP immediately before is positive or "0°, the 

jo next GOP's object bit quantity is calculated based on 
equation (6). 

[0081] At step S15, whether or not to end the image 
encoding is judged. When the encoding should con- 
tinue, the processing returns to step S4 and the next 
is GOP is encoded. 

[0082] In this way, the processing from steps S4 to 
S15 is repeated, so the input image signal is encoded 
for every GOP. 

[0083] Note that the above processing is not limited 
20 to units of GOPs. The encoding may be performed for 
example in units of pictures. Also, the steps in the 
encoding can be realized in different orders and other 
processing steps may be further added in accordance 
with need. For example, in several pictures or several 
25 GOPs immediately after starting the processing, it is 
also possible to encode an input image signal at a con- 
stant bit rate set in accordance with the object bit rate 
BR ob j by the CBR controller 23 so as to set parameters 
of the encoding in accordance with the processing 
3D result in the pictures or GOPs and to shift to encoding at 
a variable bit rate thereafter. 

[0084] Here, image encoding when the encoding 
difficulty the input image signal is changed will be 
explained. At first, assume that an input image for a 
35 while after starting the encoding has an encoding diffi- 
culty of an extent that the bit quantity becomes not more 
than the maximum bit rate BR max when encoding by 
using the quantization rates 13 and 18 calculated at 
step S2. 

40 [0085] In the encoding, the generated bit quantity 
for every MB is added up in the CBR controller 23. The 
total generated bit quantity is subtracted from the object 
bit quantity R in each picture to calculate a remaining bit 
quantity AR. Then, the CBR controller 23 reduces the 

45 CBR quantizer scale so that the remaining bit quantity 
AR comes close to "0°. After reducing the CBR quan- 
tizer scale, the VBR quantizer scale and the CBR quan- 
tizer scale are compared in the quantizer scale 
comparator 25 and the larger, as a result of the compar- 

50 ison, is provided to the image compression encoder 24 
as the base quantizer scale, thus, a large remaining bit 
quantity AR is left at the end of the next GOP quantiza- 
tion. 

[0086] Here, for example, when assuming that the 
55 generated bit quantity in the GOP is 3x1 0 6 , the remain- 
ing bit quantity AR| becomes 1 004004 with respect to 
the object bit quantity of 4004004. In an image encoder 
to which the TM5 of the normal MPEG2 is applied, the 
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above remaining bit quantity AR is to be added to the 
next GOP's object bit quantity, however in the present 
embodiment, since the remaining bit mount AR is posi- 
tive, feedback by the remaining bit quantity is not per- 
formed and the next GOP's object bit quantity R is 
obtained based on the maximum bit rate BR max as 
shown in equation (6). Namely, the next GOP's object 
bit quantity becomes 4004004. 

[0081] 

[0087] On the other hand, in the VBR controller 22, 
the bit quantity generated for every MB is subtracted 
from the remaining bit quantity RBG of the GOP calcu- 
lated based on equation (3). In the above conditions, 
when the remaining bit quantity RBG of the GOP calcu- 
lated based on equation (3) is 2002002 and the total of 
the generated bit quantity of the MBs in 1 GOP is 
3000000 (3x1 0 6 ), the remaining bit quantity at the end 
of the GOP becomes (2002002-3000000)=-997998. 
When the value is subtracted from the VBR buffer value 
VBRb, it is (1920000-(-997998))=2917998. In accord- 
ance therewith, the VBR quantizer scale VBRq in the 
next GOP can be obtained by equation (2). The 
VBRq=(2917998/(29x300))M9 stands. In this way, the 
VBR quantizer scale changes for every GOP and it is 
possible to prevent the generated bit quantity from 
largely departing from the object bit quantity. 
[0088] Next, suppose that an image having a high 
encoding difficulty which exceeds the maximum bit rate 
BRmax ' s ' n P u t witn tne VBR quantizer scale obtained 
above. In this case, the CBR quantizer scale rapidly 
increases during one picture and exceeds the VBR 
quantizer scale, so as a result of comparing the VBR 
quantizer scale VBRq and the CBR quantizer scale 
CBRq in the quantizer scale comparator 25, the larger 
CBR quantizer scale is provided as a base quantizer 
scale to the image compression encoder 24. This state 
is equal to encoding by CBR rate control at the maxi- 
mum bit rate. After a while, since the VBR quantizer 
scale becomes larger, the VBR quantizer scale VBRq is 
selected again by using the quantizer scale comparator 
25 and output to the image compression encoder 24. 
[0089] As explained above, when encoding an input 
image at a variable bit rate by one encoding operation, 
it has to be guaranteed that the maximum bit rate not be 
exceeded when the encoding difficulty is abruptly 
heightened. By using the CBR controller 23 normally 
provided in an image encoder, the VBR rate control can 
be realized without largely changing the configuration of 
the image encoder. Also, when the CBR controller 23 is 
not used as in the present embodiment, since the VBV 
buffer comes close to underflowing, the picture quality is 
deteriorated by cutting the DCT coefficient by the image 
compression encoder 24, but in the present embodi- 
ment, this can be prevented. 

[0090] The VBR buffer value VBRb is output to the 
VBR buffer memory 21 at the end of the encoding and 



stored therein. The stored value becomes an initial 
value of the VBR buffer value VBRb at the time of start- 
ing the next encoding, so the VBR buffer value VBRb 
becomes the value when completing the previous 

5 encoding. 

[0091] Figure 6 is a view of a bit rate in image 
encoding processing in an image encoder of the 
present embodiment. As shown in Fig. 6, a bit rate of an 
image gently changes about an object bit rate in accord- 

w ance with the encoding difficulty. Namely, VBR control is 
performed. When a complex image is input and the 
encoding bit rate possibly exceeds .a predetermined 
maximum bit rate prescribed by image encoding stand- 
ards, the CBR control starts functioning to control so 

is that the bit rate does not exceed the maximum bit rate. 
Due to the controlling, the image encoder of the present 
invention is capable of maintaining picture quality 
almost uniform by variably controlling the bit rate in 
accordance with the encoding difficulty of an image 

20 while observing the maximum bit rate prescribed by the 
standards. 

INDUSTRIAL APPLICABILITY 

25 [0092] As explained above, according to the image 
encoder and the method of the present invention, com- 
pression and encoding of an image signal at a variable 
bit rate by one encoding operation can be realized. Fur- 
thermore, by effectively using a general encoding cir- 
30 cuit, there is an advantage that variable bit rate 
processing can be realized in an application in which 
real-time encoding is required without largely changing 
the circuit configuration. 



image encoder, comprising: 

a first quantizer scale setting means for setting 
a first quantizer scale in a first image process- 
ing unit in accordance with an input image sig- 
nal so as to attain an object bit rate; 
a second quantizer scale setting means for set- 
ting a second quantizer scale in a second 
image processing unit which is shorter than 
said first image processing unit in accordance 
with said input image signal so as not to 
exceed a maximum bit rate prescribed in 
advance; and 

an encoding means for performing encoding 
processing on said input image signal by using 
the larger of said first and second quantizer 
scales. 

55 2. An image encoder as set forth in claim 1, wherein 
said first image processing unit is a GOP (Group Of 
Picture: an image group including a plurality of pic- 
ture) unit. 
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3. an image encoder as set forth in claim 1, wherein 
said second image processing unit is a macroblock 
unit. 

4. An image encoder, comprising: 5 

a first quantizer scale setting means for setting 
a first quantizer scale in a first image process- 
ing unit in accordance with an input image sig- 
nal so as to attain an object bit rate; io 
a second quantizer scale setting means for set- 
ting a second quantizer scale in a second 
image processing unit which is shorter than 
said first image processing unit in accordance 
with said input image signal so as not to rs 
exceed a maximum bit rate prescribed in 
advance; 

a quantizer scale selecting means for selecting 
the larger of said first and second quantizer 
scales; and 20 
an encoding means for performing encoding 
processing on said input image signal by using 
said quantizer scale selected by said quantizer 
scale selecting means. 

25 

5. an image encoder as set forth in claim 4, wherein 
said first image processing unit is a GOP (Group Of 
Picture: an image group including a plurality of pic- 
ture) unit. 

30 

6. an image encoder as set forth in claim 1, wherein 
said second image processing unit is a macroblock 
unit. 

7. An image encoder as set forth in claim 4, wherein 35 
said first quantizer scale setting means sets an 
object bit amount of encoding processing in said 
first image processing unit. 
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processing unit for every said second image 
processing unit. 

11. An image encoder as set forth in claim 4, wherein 
said second quantizer scale setting means sets an 
object bit amount in the next said second image 
processing unit in accordance with the object bit 
rate and the remaining amount of encoding bits in 
the immediately previous said second image 
processing unit when instructed to encode an 
image by said second quantizer scale. 

12. An image encoder as set forth in claim 4, wherein 
said second quantizer scale setting means sets an 
object bit amount in the next said second image 
processing unit in accordance with the maximum bit 
rate and the remaining amount of encoding bits in 
the immediately previous said second image 
processing unit when instructed to encode an 
image by said first quantizer scale. 

13. An image encoder as set forth in claim 12, wherein 
said second quantizer scale setting means sets an 
object bit amount in the next said second image 
processing unit in accordance with the remaining 
amount of encoding bits and the maximum bit rate 
when a remaining amount of encoding bits in the 
immediately previous said second image process- 
ing unit is negative. 

14. An image encoder as set forth in claim 12, wherein 
said second quantizer scale setting means sets an 
object bit amount in the next said second image 
processing unit in accordance with the maximum bit 
rate and regardless of the remaining amount of 
encoding bits when a remaining amount of encod- 
ing bits in the immediately previous said second 
image processing unit is positive or "0". 



8. An image encoder as set forth in claim 4, wherein 40 
said first quantizer scale setting means sets said 
first quantizer scale in accordance with a remaining 
amount of encoding bits and a response speed in 

the immediately previous said first image process- 
ing unit. 45 

9. An image encoder as set forth in claim 4, compris- 
ing a memory means for storing the remaining 
amount of encoding bits in said first image process- 
ing unit; and wherein said first quantizer scale set- so 
ting means sets an object bit amount in the next 
said first image processing unit in accordance with 

the remaining amount of encoding bits stored in 
said memory means. 

55 

10. An image encoder as set forth in claim 4, whrein 
said second quantizer scale setting means sets an 
object bit amount in the next said second image 



15. An image encoder as set forth in claim 4, further 
comprises an instruction signal for instructing to 
perform image encoding processing by using either 
one of the first quantizer scale or a second quan- 
tizer scale, and a control means for respectively 
outputting the object bit rate and the maximum bit 
rate to said first quantizer scale setting means and 
said second quantizer scale setting means. 

16. An image encoder as set forth in claim 15, wherein 
said control means outputs said instruction signal 
to said quantizer scale selecting means. 

17. An image encoder as set forth in claim 15, wherein 
said control means outputs said instruction signal 
to instruct to perform processing by said second 
quantizer scale in said first processing unit on said 
input image signal when instructing image encod- 
ing processing by said first quantizer scale. 



40 



45 



10 



19 



EP 1 091 588 A1 



20 



18. An image encoder as set forth in claim 17, wherein 
said control means outputs said instruction signal 
to instruct to switch to image encoding processing 
by said first quantizer scale when image encoding 
processing by said second quantizer scale in said 
first image processing unit is completed. 

19. An image encoding method, including the steps of: 

setting a first quantizer scale oin a first image 
processing unit in accordance with an input 
image signal so as to attain an object bit rate; 
setting a second quantizer scale in a second 
image processing unit which is shorter than 
said first image processing unit in accordance 
with said input image signal so as not to 
exceed a maximum bit rate prescribed in 
advance; and 

performing encoding processing on said input 
image signal by using the larger of said first 
and second quantizer scales. 

20. An image encoder as set forth in claim 19, wherein 
an object bit amount in said first image processing 
unit is set and said first quantizer scale is set in 
accordance with a remaining amount of encoding 
bits and response speed in the immediately previ- 
ous said first image processing unit when perform- 
ing image encoding processing by using said first 
quantizer scale. 

21. An image encoding means as set forth in claim 19, 
wherein if said first quantizer scale calculated in 
accordance with the remaining amount of encoding 
bits and response speed in the immediately previ- 
ous said first image processing unit is smaller than 
a reference value, the reference value is set as a 
quantizer scale in the next said first image process- 
ing unit when performing image encoding process- 
ing by using said first quantizer scale. 

22. An image encoding method as set forth in claim 19, 
wherein said first image processing unit is a GPO 
unit. 

23. An image encoding method as set forth in claim 19, 
wherein said second image processing unit is a 
macroblock unit. 



an object bit rate and a remaining amount of 
encoding bits in the immediately previous said 
second image processing unit when instructed 
to perform image encoding processing by said 

5 second quantization scale; and 

an object bit amount in the next said first image 
processing unit is set in accordance with a 
maximum bit rate and the remaining amount of 
encoding bits in the immediately previous said 

10 first image processing unit if the remaining 

amount of encoding bits in the immediately pre- 
vious said first image processing unit is nega- 
tive, while, an object bit amount in the next said 
first image processing unit is set in accordance 

15 with said maximum bit rate regardless of the 

immediately previous said first image process- 
ing unit if the remaining amount of encoding 
bits in the immediately previous said first image 
processing unit when instructed to perform 

20 image encoding processing by said first quan- 

tizer scale. 

25. An image encoding method as set forth in claim 19, 
for performing image encoding processing by said 

25 second quantizer scale in said second image 
processing unit on said input image signal, and 
then, performing image encoding processing by 
said first quiantizer scale in accordance with a 
result of the encoding processing by the second 

30 quantizer scale when performing image encoding 
processing by said first quantizer scale 



24. An image encoding method as set forthin claim 1 9, so 
wherein: 



an object bit amount in the next said second 
image processing unit is set for each of said 
second image processing units when deter- 55 
mining said second quantizer scale 
an object bit amount in the next said second 
image processing unit is set in accordance with 
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